Recent Posts

numpy.random.choice without duplication is slow for large number

This is based on the issue at tf2rl. To make non-duplicated uniform pickup, numpy.random.choice(a, size, replace=False) is used. I learned this is quite slow when a (sampling space size) is

Introduce new Python package HashDL

I released new Python package HashDL ( PyPI, mirror). This package is non-official third-party reinplementation of SLIDE1. Hopefully soon, I will evaluate performance. B. Chen et al., “SLIDE : In

cpprb v10.1.0 can save & load transitions

From cpprb version 10.1.0, ReplayBuffer and its sub-classes can save and load transitions. from cpprb import ReplayBuffer rb1 = ReplayBuffer(256, {"obs": {"shape": 3}, "act": {}, "rew": {}, "done": {}}, next_of="obs")

GitLab CI/CD: Single job for multiple conditions

At a previous post, I used following configuration to achive manual or scheduled job at GitLab CI/CD. .docker_build_base: &docker_build image: docker:latest stage: build_image services: - docker:dind script: - docker login

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

When import cpprb, I got ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject. Recently, NumPy has changed its ABI at version




Bayes Neural Network for TensorFlow

Gym Notebook Wrapper

Wrapper for running and rendering OpenAI Gym on Jupyter Notebook


TensorFlow2 Reinforcement Learning


Replay buffer classes for reinforcement learning.


Survey of Experience Replay

Survey of Experience Replay

Hiroyuki Yamada


Hiroyuki Yamada got interested in IT (including programming, PC, etc.) during his graduate school. A few years later, he became an IT researcher at a Japanese company.

Encouraged by a friend, he started development of reinforcement learning software cpprb using Cython as a hobby at the beginning of 2019. This continuing development is still one of the biggest hobby in his life.

The author started this personal blog at the end of 2018. The motivation was to learn the way of making a site by utilizing GitLab CI/CD and a static site generator Hugo. Additionaly, he wanted to organize his own knowledge and experience based on the cpprb development.

H. Yamada’s GitHub Stats


  • Artificial Intelligence
  • Cloud Native Computing
  • C++
  • Python


Data Science

Statistics/Machine Learning


C++/Python (Cython)/shell script/JavaScript/Emacs Lisp/C#

Cloud Native Computing